Testing Concurrent Object-Oriented Systems with Spec Explorer

We describe a practical model-based testing tool developed at Microsoft Research called Spec Explorer. Spec Explorer enables modeling and automatic testing of concurrent object-oriented systems. These systems take inputs as well as provide outputs in form of spontaneous reactions, where inputs and outputs can be arbitrary data types, including objects. Spec Explorer is being used daily by several Microsoft product groups. The here presented techniques are used to test operating system components and Web service infrastructure. Transition Systems Formalize Reactive and Distributed Systems. Reactive and distributed systems are inherently nondeterministic. No single agent (component, thread, etc.) controls all state transitions, and even external entities like the operating systems scheduler or the network may play a role. A practical and theoretically sound way to test the evolution of semi-independent state spaces is to use a kind of transition system known as an interface automaton [3]. Interface automata make a distinction between input transitions and output transitions. In some states, input is enabled and we can drive the system forward by giving it new things to do; at other times the system and its environment choose what happens next. This is like a game where players take turns. Sometimes it is our turn to make a move; sometimes it is the systems. To illustrate how this works we will use a network-based chat system as an example. In the chat system there are multiple clients that may post messages. The system delivers pending messages in FIFO order with local consistency. Figure 1 shows a typical scenario of the chat systems’ behavior as an interface automaton. The nodes of the graph represent distinct states of the system. The arcs represent actions that change the systems state. Each state in the graph is either input enabled or output enabled. The states drawn with ovals represent active, input-enabled states where a client may give the system new work to do. States drawn with diamonds are passive, output-enabled states where the system reacts to input or spontaneously makes a move of its own choosing. The Post action is said to be controllable because it can be invoked by a user to provide system input. The Deliver action is only observable; that is, it is an output message. The names of observable actions in the graph are prefixed by the ? symbol. Note that in some passive states there is a race between what the user may do and what the system may do. The Timeout transition, here represented by a transition that carries no label, indicates that no output was seen in the time the user was willing to wait. This causes a transition from an output-enabled state to an input-enabled state. J.S. Fitzgerald, I.J. Hayes, and A. Tarlecki (Eds.): FM 2005, LNCS 3582, pp. 542–547, 2005. c ©Springer-Verlag Berlin Heidelberg 2005 Testing Concurrent Object-Oriented Systems with Spec Explorer 543 [c1->[c0->[]], [c0->[c1->[]] [c1->[c0->["hi"]], c0->[c1->[]] [c1->[c0->"hi"]], [c0->[c1->[]] ?Deliver("hi", c0, c1) [c1->[c0->"hi"]], [c0->[c1->"hi"]] Post(c1, "hi") Post(c0, "hi") [c1->[c0->[]]}, [c0->[c1->"hi"]] Post(c1, "hi") ?Deliver("hi", c1, c0) [c1->[c0->[]], [c0->[c1->["hi"] ?Deliver("hi", c1, c0) ?Deliver("hi", c0, c1)